Skip to content

Support timeToLive attributes in DynamoDb Enhanced Client#6152

Open
roamariei wants to merge 2 commits intoaws:masterfrom
roamariei:feature/support-timeToLive-attributes-in-dynamodb-enhanced-client
Open

Support timeToLive attributes in DynamoDb Enhanced Client#6152
roamariei wants to merge 2 commits intoaws:masterfrom
roamariei:feature/support-timeToLive-attributes-in-dynamodb-enhanced-client

Conversation

@roamariei
Copy link
Contributor

@roamariei roamariei commented Jun 3, 2025

Description

Support TimeToLive attributes in DynamoDb Enhanced Client

Motivation and Context

#4295

Modifications

Added DynamoDbTimeToLiveAttribute annotation, TimeToLive extension to run before every write, added new operations DescribeTimeToLive and UpdateTimeToLive in enhanced client.

Testing

Added unit tests for TimeToLiveExtension, DescribeTimeToLiveOperation, UpdateTimeToLiveOperation and integration tests for update timeToLive with enhanced client in TimeToLiveRecordTest.

Test Coverage Checklist

Scenario Done Comments if Not Done
1. Different TableSchema Creation Methods Not impacted
a. TableSchema.fromBean(Customer.class) [ ]
b. TableSchema.fromImmutableClass(Customer.class) for immutable classes [ ]
c. TableSchema.documentSchemaBuilder().build() [ ]
d. StaticTableSchema.builder(Customer.class) [ ]
2. Nesting of Different TableSchema Types Not impacted
a. @DynamoDbBean with nested @DynamoDbBean as NonNull [ ]
b. @DynamoDbBean with nested @DynamoDbImmutable as NonNull [ ]
c. @DynamoDbImmutable with nested @DynamoDbBean as NonNull [ ]
d. @DynamoDbBean with nested @DynamoDbBean as Null [ ]
e. @DynamoDbBean with nested @DynamoDbImmutable as Null [ ]
f. @DynamoDbImmutable with nested @DynamoDbBean as Null [ ]
3. CRUD Operations
a. scan() [ ]
b. query() [ ]
c. updateItem() [ ]
d. putItem() [x]
e. getItem() [x]
f. deleteItem() [ ]
g. batchGetItem() [ ]
h. batchWriteItem() [ ]
i. transactGetItems() [ ]
j. transactWriteItems() [ ]
4. Data Types and Null Handling Not impacted
a. top-level null attributes [ ]
b. collections with null elements [ ]
c. maps with null values [ ]
d. conversion between null Java values and AttributeValue [ ]
e. full serialization/deserialization cycle with null values [ ]
5. AsyncTable and SyncTable
a. DynamoDbAsyncTable Testing [ ]
b. DynamoDbTable Testing [x]
6. New/Modification in Extensions
a. Tables with Scenario in ScenarioSl No.1 (All table schemas are Must) [x]
b. Test with Default Values in Annotations [x]
c. Combination of Annotation and Builder passes extension [x]
7. New/Modification in Converters Not impacted
a. Tables with Scenario in ScenarioSl No.1 (All table schemas are Must) [ ]
b. Test with Default Values in Annotations [ ]
c. Test All Scenarios from 1 to 5 [ ]

Screenshots (if appropriate)

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)

Checklist

  • I have read the CONTRIBUTING document
  • Local run of mvn install succeeds
  • My code follows the code style of this project
  • My change requires a change to the Javadoc documentation
  • I have updated the Javadoc documentation accordingly
  • I have added tests to cover my changes
  • All new and existing tests passed
  • I have added a changelog entry. Adding a new entry must be accomplished by running the scripts/new-change script and following the instructions. Commit the new file created by the script in .changes/next-release with your changes.
  • My change is to implement 1.11 parity feature and I have updated LaunchChangelog

License

  • I confirm that this pull request can be released under the Apache 2 license

@roamariei roamariei force-pushed the feature/support-timeToLive-attributes-in-dynamodb-enhanced-client branch 3 times, most recently from 28f6637 to 5206b59 Compare June 6, 2025 07:02
@roamariei roamariei marked this pull request as ready for review June 6, 2025 07:02
@roamariei roamariei requested a review from a team as a code owner June 6, 2025 07:02
@anasatirbasa anasatirbasa force-pushed the feature/support-timeToLive-attributes-in-dynamodb-enhanced-client branch from 9b7a52a to 420e193 Compare December 14, 2025 19:33
@anasatirbasa anasatirbasa force-pushed the feature/support-timeToLive-attributes-in-dynamodb-enhanced-client branch from 97468cb to 2291714 Compare January 11, 2026 15:11
@andreas-grafenberger andreas-grafenberger force-pushed the feature/support-timeToLive-attributes-in-dynamodb-enhanced-client branch from 2291714 to 207b147 Compare February 17, 2026 08:21
@andreas-grafenberger andreas-grafenberger force-pushed the feature/support-timeToLive-attributes-in-dynamodb-enhanced-client branch from 7673952 to fa8d740 Compare March 9, 2026 18:07
@andreas-grafenberger
Copy link
Contributor

Cannot edit the PR description, so I'll attach here the test coverage evidence:
image

Comment on lines +148 to +150
public String baseField;
public long duration;
public ChronoUnit unit;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should these be public or private ?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think these should be private and final to easily ensure the @ThreadSafe is respected.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree, updated from public to private final.

@ThreadSafe
public final class TimeToLiveExtension implements DynamoDbEnhancedClientExtension {

public static final String CUSTOM_METADATA_KEY = "TimeToLiveExtension:TimeToLiveAttribute";

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1- Consider renaming this to something like, TTL_ATTRIBUTE_METADATA_KEY, current name is too vague
2- This Key is under @SdkPublicApi class, because the class is @SdkPublicApi, everything public on it becomes part of the public API contract, but it is only used from UpdateTimeToLiveOperation which is an SdkInternalAPI for any backward compatibility issues going forward. Consider either making the field package-private (so UpdateTimeToLiveOperation in the same package can still access it), or moving the constant to an @SdkInternalApi shared utility class...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Agree, but this is the pattern used for the other extensions.
  2. I would go with the last approach, and create an @SdkInternalApi shared utility class - and place there all *_ATTRIBUTE_METADATA_KEY for extensions. Would make sense? wouldn't make too much sense to create the class only for TimeToLive extension attributes... what do you think?

Comment on lines +174 to +183
public Consumer<StaticTableMetadata.Builder> modifyMetadata(String attributeName,
AttributeValueType attributeValueType) {
Map<String, Object> customMetadataMap = new HashMap<>();
customMetadataMap.put("attributeName", attributeName);
customMetadataMap.put("baseField", baseField);
customMetadataMap.put("duration", duration);
customMetadataMap.put("unit", unit);

return metadata -> metadata.addCustomMetadataObject(CUSTOM_METADATA_KEY,
Collections.unmodifiableMap(customMetadataMap));

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BeanTableSchema validates that only one TTL attribute exists, but StaticTableSchema has no such guard. Since modifyMetadata stores a Map (not a Collection) via addCustomMetadataObject, a second TTL attribute silently replaces the first with no error:

StaticTableSchema.builder(MyItem.class)
    .addAttribute(Long.class, a -> a.name("ttl1")
        .getter(MyItem::getTTL1).setter(MyItem::setTTL1)
        .tags(timeToLiveAttribute("createdAt", 30, ...)))
    .addAttribute(Long.class, a -> a.name("ttl2")
        .getter(MyItem::getTTL2).setter(MyItem::setTTL2)
        .tags(timeToLiveAttribute("updatedAt", 7, ...)))
    .build();
// ttl1 config is silently lost, only ttl2 is tracked

DynamoDB only supports one TTL attribute per table, so this is always a user error, but the SDK should fail loudly rather than silently dropping one.

Recommendation: You may consider moving the validation into TimeToLiveAttribute.modifyMetadata() so it works for all schema types

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree, the validation is applied only on BeanTableSchema and I am preparing parameterized tests to cover the remaining schemas with validation updated - tests still in progress, have pretty many scenarios to cover with different record types.

Also, I feel like addCustomMetadataObject is not consistent in terms of validation:

       private void mergeCustomMetaDataObject(String key, Object object) {
            if (object instanceof Collection) {
                this.addCustomMetadataObject(key, (Collection<Object>) object);
            } else if (object instanceof Map) {
                this.addCustomMetadataObject(key, (Map<Object, Object>) object);
            } else {
                this.addCustomMetadataObject(key, object);
            }
        }

because the else branch throws an exception when finds duplicates:

        public Builder addCustomMetadataObject(String key, Object object) {
            if (customMetadata.containsKey(key)) {
                throw new IllegalArgumentException("Attempt to set a custom metadata object that has already been set. "
                                                   + "Custom metadata object key: " + key);
            }

            customMetadata.put(key, object);
            return this;
        }

TimeToLiveExtension.builder().build();
private static final List<DynamoDbEnhancedClientExtension> DEFAULT_EXTENSIONS =
Arrays.asList(DEFAULT_VERSIONED_RECORD_EXTENSION, DEFAULT_ATOMIC_COUNTER_EXTENSION);
Arrays.asList(DEFAULT_VERSIONED_RECORD_EXTENSION, DEFAULT_ATOMIC_COUNTER_EXTENSION, DEFAULT_TIME_TO_LIVE_EXTENSION);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This means every existing DynamoDbEnhancedClient using default extensions will now run TimeToLiveExtension.beforeWrite() on every write operation, even for tables
with no TTL attributes. The extension does an early return when no metadata is found, but we still change the behavior:

  1. Performance: An additional extension in the chain for every write across the entire SDK, for all users, even those who never use TTL. The customMetadataObject() lookup + Optional.orElse(null) runs on every putItem/updateItem/batchWriteItem/transactWriteItems.

  2. Double registration: Users who extend the defaults will register TTL twice:

   // This is how TimeToLiveRecordTest in this PR does it — TTL extension runs twice per write
   DynamoDbEnhancedClient.builder()
       .extensions(Stream.concat(
           ExtensionResolver.defaultExtensions().stream(),
           Stream.of(timeToLiveExtension))
           .collect(Collectors.toList()))
       .build();

Should we think about making this extension opt-in instead ?

also, if we want to keep this, The Javadoc comment on defaultExtensions() says "Currently this is just the VersionedRecordExtension", we may need to update it

default DescribeTableEnhancedResponse describeTable() {
throw new UnsupportedOperationException();
}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

consider adding javadoc, also applies to UpdateTimeToLiveEnhancedResponse and to Async table

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

javadoc added

Comment on lines +100 to +102
if (baseValue instanceof LocalTime) {
return LocalDate.now().atTime((LocalTime) baseValue).plus(duration, unit).toEpochSecond(ZoneOffset.UTC);
}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to support local time ?

if (baseValue instanceof LocalTime) {
    return LocalDate.now().atTime((LocalTime) baseValue).plus(duration, unit).toEpochSecond(ZoneOffset.UTC);
}

two potential problems:

  1. Midnight jump: LocalDate.now() defines what "today" means, and that changes at midnight. Items with the same LocalTime base value and duration get TTLs that differ by a day depending on which side of midnight the write lands:
  • Write at 11:59 PM March 10 -> LocalDate.now() = March 10-> TTL = March 11 00:05 (expires in ~6 minutes)
  • Write at 12:01 AM March 11 -> LocalDate.now() = March 11-> TTL = March 12 00:05 (expires in ~24 hours)
  1. JVM timezone dependency: LocalDate.now() uses ZoneId.systemDefault(), but the epoch conversion uses ZoneOffset.UTC. If the JVM runs in America/New_York, LocalDate.now() could return a different date than UTC, shifting the TTL by hours depending on where the server runs.

If you say, LocalDate.now(ZoneOffset.UTC), it would fix problem 2, but problem 1 is still there to LocalTime having no date component, there's always a midnight boundary where the TTL jumps by 24 hours.

Recommendation: You may consider removiong LocalTime support and let it fall through to the existing IllegalArgumentException if that's acceptable. The other five types (Instant, LocalDate, LocalDateTime,
ZonedDateTime, Long) are all deterministic and cover every real TTL use case. Users who need time-of-day-based TTL can compute the epoch seconds themselves with explicit control over the date and timezone.

Copy link

@valeriodelbello-amazon valeriodelbello-amazon Mar 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I strongly suggest we only use UTC and only support Instant with a Clock parameter like in AutoGeneratedTimestampRecordExtension.

Comment on lines +70 to +72
String baseFieldName = (String) customTTLMetadata.get("baseField");
Long duration = (Long) customTTLMetadata.get("duration");
TemporalUnit unit = (TemporalUnit) customTTLMetadata.get("unit");

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If any key is missing or has a wrong type (like someone manually constructs metadata via StaticTableMetadata.builder().addCustomMetadataObject(...)), these lines will throw NullPointerException with no context about which key failed or why. Can consider to use Validate.notNull() with descriptive messages..

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree and a wrong type would throw a different exception and require a dedicated check.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added validation.


private final boolean enabled;

public UpdateTimeToLiveOperation(boolean enabled) {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this supposed to be private ?

*
* @return The properties of the timeToLive specification.
*/
public TimeToLiveSpecification table() {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

table() function name is misleading, may consider renaming it to something like timeToLiveSpecification() ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated, should be fine now.

import software.amazon.awssdk.utils.Validate;

/**
* Defines the elements returned by DynamoDB from a {@code DescribeTimeToLive} operation, such as

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess not DescribeTimeToLive , probably mistake ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, updated.

import software.amazon.awssdk.enhanced.dynamodb.mapper.annotations.DynamoDbPartitionKey;

@DynamoDbBean
public class RecordWithDefaultTTL {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this and RecordWithSimpleTTL dead code?

builder.attributes(attributes);

if (ttlAttributesCount.intValue() > 1) {
throw new IllegalArgumentException(

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should cover this in the unit tests.

@valeriodelbello-amazon
Copy link

Should we add TTL validation to ImmutableTableSchema too?


private TimeToLiveAttribute(String baseField, long duration, ChronoUnit unit) {
this.baseField = baseField;
this.duration = duration;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we check this is not negative somewhere?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added validation for duration in modifyMetadata and beforeWrite, I will have a look if there is a better place for this validation.

if (customTTLMetadata == null) {
throw new IllegalArgumentException("Custom TTL metadata object is null");
}
String ttlAttributeName = (String) customTTLMetadata.get("attributeName");
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we not using validateMetadataValue helper method ?

Map<String, ?> customTTLMetadata = tableSchema.tableMetadata()
.customMetadataObject(CUSTOM_METADATA_KEY, Map.class).orElse(null);
if (customTTLMetadata == null) {
throw new IllegalArgumentException("Custom TTL metadata object is null");
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this Message is Confusing a bit, it exposing Internal Logic, instead of helping caller to help fix the issue.

It should be more readable to caller having something like No TTL attribute is configured in the table schema for table, Use @DynamoDbTimeToLiveAttribute on a Long getter....

@Retention(RetentionPolicy.RUNTIME)
@BeanTableSchemaAttributeTag(TimeToLiveAttributeTags.class)
@SdkPublicApi
public @interface DynamoDbTimeToLiveAttribute {
Copy link

@ysunio ysunio Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Classes / Interfaces should have @ since tags

long duration = validateMetadataValue(customTTLMetadata, "duration", Long.class);
TemporalUnit unit = validateMetadataValue(customTTLMetadata, "unit", TemporalUnit.class);

Validate.isTrue(duration >= 0, "Custom TTL metadata key 'duration' must not be negative.");
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should add baseFieldName check as well, here.
What happens if User does @DynamoDbTimeToLiveAttribute(duration=30, unit=DAYS) — forgot baseField.

It would be silent no-op when duration > 0 and baseField="".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants